Dual-Memory Deep Learning Architectures for Lifelong Learning of Everyday Human Behaviors

Learning from human behaviors in the real world is important for building human-aware intelligent systems such as personalized digital assistants and autonomous humanoid robots. Everyday activities of human life can now be measured through wearable sensors. However, innovations are required to learn these sensory data in an online incremental manner over an extended period of time. Here we propose a dual memory architecture that processes slow-changing global patterns as well as keeps track of fast-changing local behaviors over a lifetime. The lifelong learnability is achieved by developing new techniques, such as weight transfer and an online learning algorithm with incremental features. The proposed model outperformed other comparable methods on two real-life data-sets: the image-stream dataset and the real-world lifelogs collected through the Google Glass for 46 days.